The English activist and poet John Milton once wrote, “The mind is its own place, and in it self, can make a Heav’n of Hell, a Hell of Heav’n.”1 He composed these words in his epic poem Paradise Lost . Through his writings, Milton gave us one of the finest observations about human perception, as well as UX: what we experience is what we perceive.
Paradise Lost is an unsurprising title when you consider what happened in the year it was published. In 1667, Dutch ships sailed up the English River Thames, bombarded towns, burned a dozen vessels, stole the English flagship, and towed it back home to set it up as a tourist attraction.2 These were the days of unparalleled turmoil. In less than a decade, three civil wars shifted the balance from monarchy to commonwealth to protectorate. Religious strife bled into the politics and political turmoil, igniting religious fervor across England, Scotland, and Ireland. The king dissolved one parliament and fought another. In turn, parliament executed one king and exiled a second. You could not blame Milton for his relativism. He witnessed war, as well as peace; corruption, as well as charity; depression, as well as prosperity. He was also blind.
Milton did not see the world; however, he certainly perceived it. Our senses are only one way we perceive. We sense the world through our eyes, ears, nose, mouth, and skin. We see a fire’s flame, hear its roar, smell and taste its smoke, and feel its warmth. Our minds shape these senses into the perception of fire. Psychologists call this bottom-up processing. Conversely, our minds also form perceptions based on prior experiences, general concepts, and expectations. We can look up to the heavens and see stars twinkle against an endlessly black sky. Some of us will see constellations of gods, animals, and objects based on our prior exposure to similar patterns. Psychologists call this top-down processing.
What remains is a perception. Senses transform into fires. Stars transform into gods. Likewise, pixels and screen layouts, beeps and button clicks, swipes and finger gestures, help form our perceptions of digital applications. A dramatic visual design connotes excitement, whereas a zany sound may imply childlike wonder. Your expectations of a game app may align with such senses although a spreadsheet application may not. This is our perceptual processing at work. Understanding how perception works helps improve user experiences. It makes bad software better. It makes good software great. After all, it can make heaven out of hell.
Top-Down Processing
The Viking 1 Orbiter, a NASA spacecraft, took a curious photo when it flew past the Martian surface in 1976. The image, taken by its two vidicon tube cameras, showed a landscape pockmarked with craters and shadowed by mesas. The mostly featureless region of Mars named Cydonia was nothing extraordinary, excepting the humanoid face staring up from its surface. First popularized by the book, The Monuments of Mars: A City on the Edge of Forever , by Richard C. Hoagland, this image became known as the “Face of Mars.” The covers of grocery aisle tabloids have celebrated the image ever since.
In actuality, the “Face of Mars” is a rather ordinary mesa reaching 800 feet high.4 And, like any tall thing under sunlight, it casts shadows. When photographed, the broad shadows it created looked like a jawline, deep shadows looked like eye sockets, and small shadows looked like a nose.
Believing the “Face of Mars” is an alien glamour shot or a garden-variety rock pile is a secondary issue. The primary question is why do most of us see it as a face? Our answer lies in a discussion of schemas.
A schema is a mental shortcut, a way to interpret incomplete information. Say you are gazing up at the stars on a clear evening. You marvel at the expanse of the universe and see a flickering light. For a fleeting moment, you witness a few dim flashes streak across the dark canvas of the night sky. You might think it is a star, a plane, or a UFO. However, there is a good chance that the dim flashes are a passing satellite . Many people are unaware that you can view satellites, including the International Space Station, with the naked eye.5 Our preconception—our schema—for a light in the sky doesn’t include satellites, but we do have a schema for shining stars, passing planes, and even UFOs. Your schema for a star may have started when a parent pointing to the heavens said, “Look sweetie, that’s a star.” The first time you viewed a plane passing in the night sky, you likely confused it for a star, because your schema for stars was already well established. Later in life, your schema for planes became established, as well. Upon reading this chapter, your schema for unknown lights in the sky now includes satellites, if it didn’t already.
People have a strong schema for human faces. We even have a place within our brains dedicated to processing faces, the fusiform face area (FFA) . Whereas the brain’s visual cortex processes every other visual stimulus (from paper clips to rocket ships), the FFA gains an efficiency through its unique role. A 2009 fMRI study showed that humans can recognize a face in 130 milliseconds,6 roughly half the amount of time it takes to blink your eye. Studies show that four-month-old infants process faces almost as quickly as adults.7 The noted scientist, Carl Sagan, hypothesized in his book The Demon-Haunted World: Science as a Candle in the Dark that humans evolved a hyper awareness of faces to recognize the emotional states of humans and other animals. The smiling face of a parent indicated a safe opportunity to bond, whereas the snarling face of a predator indicated a strong warning to flee.
From sunup to sundown, our minds sort the world into a series of perceptions. Schemas shape these perceptions, like the gravitational forces of nearby planets, pulling in everything from within their orbits. Some perceptions burn up upon reentry, while others land and become the foundations of new ideas.
Mental Models
In retrospect, the contest afforded the storeowner a free merchandise display; but I received much more in return: for a few days, I mentally transported myself into the seat of a P51 Mustang. Although nearly 40 years have passed, I still remember those daydreams of flying over the countryside of Beavercreek, Ohio.
Surely, we have all had similar experiences when growing up. You imagined being an astronaut, winning Miss America, or living the life of a gunslinger. Yet, just as a nerdy kid like me had no background in flying fighter-bombers, it is likely that you had no direct connection to any of those imagined pursuits. So how could we imagine them so clearly?
We form mental models of perceived experiences. Like flying a toy plane, we construct these models based on our related, past experiences: the books we’ve read, the movies we’ve watched, the conversations we’ve had. We build mental models of software much in the same way.
If a mental model sounds like a schema , take comfort that you are not alone in assuming this. People often confuse the two; however, they differ in a few key ways. Mental models include schemata , but also behaviors and outcomes. Whereas a schema may describe a plane, a mental model describes a plane, as well as flying and landing it.
Mental models set expectations. However, expectations are merely contours that we can sculpt into other forms. New experiences need not be strict combinations of past ones, but mental models are often where new ideas take flight.
Just Noticeable Differences
You are a frog. You sit happily in a cool pot of water. You have not a care in the world. The water starts to warm. It is rather pleasant, reminding you of your time in Palm Springs. The water grows warmer. You look around, admiring the sturdy craftsmanship of your new metal home. The water grows warmer. You relax, reflecting on your life as an amphibian. The water grows warmer. You enjoy your unplanned sauna. The water boils... You are no more.
Why didn’t you jump out? One minute you basked in delight; the next minute you boiled in disbelief. You may have escaped if the temperate changes had been more noticeable. Psychophysicists call such a change a “just noticeable difference.”9 A just noticeable difference, or JND, is a unit of measurement. It describes the smallest detectable difference between two levels of stimuli.
Weber’s Law
The German professor and experimental psychologist, Ernst Heinrich Weber , first codified JNDs in the 1800s through his research on human touch and sensory physiology.10 He noted that people notice the relative change in stimuli, not the absolute. He experimented with everything from differently weighted blocks to fluctuating musical notes, but today his law is best demonstrated by your television.
TV commercials often play at a higher volume than the scheduled programming. Your ability to notice this change depends on the volume of your television: low- and medium-volume settings highlight the difference, while high-volume settings mask the difference. You notice the fluctuation between a quiet TV episode and loud commercial because of the relative difference between a low and a high volume is great. You don’t notice the fluctuation between a loud TV episode and a slightly louder commercial because the relative difference between the two volumes is small.
Coincidentally, the fluctuating volume between TV programming and commercials was so noticeable that it led to many complaints to the Federal Communications Commission , culminating in the Commercial Advertisement Loudness Mitigation (CALM) Act of 2010.11 To bypass the legislation, some commercials now fluctuate volumes within themselves to achieve a consistent average volume, while still having the same effect.
Marketing JNDs
Marketers use JNDs to determine pricing and discounts. Imagine walking past a storefront that offered a “5% Off Sale!” Not only would you continue to walk past the store, but also you would hardly even acknowledge the offer. Retailers often need to move percentage discounts into the 20-25% territory in order to attract the attentions of consumers. Marketers focus on JNDs because they wish to grab your attention without leaving their money on the table. If a consumer responds to a 20% discount, why offer 25%? Doing so would give away the additional 5% for no reason. Some companies—and even entire industries—become trapped by offering inflated JNDs. Discounts of 50%, 60%, and 70% off are viewed with as much skepticism as enthusiasm. When was the last time you saw a furniture store that was not having a “Going out of Business Sale”? Because these JNDs are so extreme, the just noticeable difference is barely noticeable at all.
Information Design JNDs
Edward Tufte, noted statistician and Yale professor, further extended JNDs within the study of information design. His “small multiples”12 demonstrated differences among similar graphs when placed next to one another. When nearly all the information is the same, the differences are easily observed. Tufte’s beautifully crafted books include numerous examples of small multiples, ranging from fly-fishing lures to Japanese calligraphy.
So far in this chapter, we have discussed how human beings use past experiences, concepts, and expectations to understand the world. But this is only half of the equation. Top-down mental processing alone can lead us down blind alleys. To understand the other half of the equation , bottom-up processing, we will exchange our blind alley for a dark one.
Bottom-Up Processing
You find yourself walking down an alleyway. It is late. It is dark. You spent the evening with friends at a downtown restaurant. Your car is parked a few streets away . Hulking silhouettes of trash dumpsters and piles of debris line the pathway, as the stench of unfinished entrees and discarded mop water fills the air. You navigate past each starlit obstacle and hear a noise. A footstep. Another follows. You speed up. So do the footsteps. Your shoulders tighten. Your stride widens. Your pulse grows. You gasp and turn and see nothing. It was merely an echo.
Stories of dark alleys are clichés and staples of horror films. The sights, smells, and sounds lead characters into states of fear and panic. Shadows transform into monsters, stenches instill dread, and unidentified footsteps indicate malicious pursuit. In actuality, the protagonists of such stories should embrace a top-down approach to their thinking: we live in the safest time in all human history, therefore what we sense is rarely danger. The 2014 UN Office on Drugs and Crime report states that we have a 1 in 16,000 chance of being murdered.13 Considering the average 30-year-old person has greater than one in five chance of living to 100,14 it would be more rational to worry about your retirement savings, rather meeting your demise in a dark alley.
But our senses can surprise even the most logical of us. Opposed to top-down processing, where we construct perceptions based on general concepts, bottom-up processing constructs perceptions based on sensory data. Listen up. I smell something fishy. This feels squishy. Watch out! Sound, smell, sight, and touch construct perceptions upwards. Sometimes senses aid us; sometimes they trick us. Like victims in a horror movie, we only realize our perceptual errors when it is too late. Yet, we can also leverage bottom-up processing to improve experiences, through which senses become tools to accelerate behaviors , assuage fears, and satisfy users.
Gestalt Grouping
Red represents everything from love to anger, from safety to danger, from Christmas to communism. A red heart shows affection. A red hand shows revolution. A red light tells us to stop. A red exit sign tells us to go. In every case, we sense red in the exact same way. Its wavelength always measures around 650 nanometers.15 Yet, how we perceive color changes. The conflict between what we sense and what we perceive lies at the core of user experience. We possess a myriad of senses—sight, sound, smell, taste, touch, temperature, and pressure. However, our perception of these senses is often a matter of gestalt.
A favorite word bandied about at dinner parties and in film schools, gestalt describes the whole of something, as opposed to its parts. When you watch a movie, you see action as a continuous motion. One frame at a time, movie stars dangle each other off the bows of ships, avatars fly among floating mountains, and toys tell you their stories. You see this motion in 24, 25, or 48 frames per second. Psychologist call this effect “apparent motion.” You experience each frame (each part), but you also experience the movement (the whole). You experience a gestalt grouping.
Gestalt grouping laws describe how people perceive objects as organized patterns. The laws cover similarity, proximity, continuity, and closure. Of these, two are especially helpful in user experience: proximity and similarity.
Proximity
We perceive objects placed next to one another as a group. Our ancestors looked into the night sky and found asterisms and constellations—groups of stars resembling eagles, charioteers, crabs, harps, gods, and dragons. We consciously and subconsciously group together the items we see throughout the day and night. Whereas we may intentionally group stars twinkling in the northern sky as the Big Dipper, we unintentionally group strangers walking down a sidewalk as being acquaintances.
Company Name:
First Name:
Last Name:
Is the “First Name” related to the “Company Name” or to the “Last Name”? Due to its proximity, users may associate the first name to the company, rather than last name.
The greatest separation happens when designers scatter information across multiple screens. Doing so destroys gestalt groupings. A designer may forget that only she knows what appears next within an application. If a designer describes something behind a curtain, it will remain a mystery to users until that curtain is opened. A designer has prior knowledge; a user does not. Users need to see information themselves, be it a confirmation page, an error screen, or the next step in a process. Unseen information is not information at all. Dense and elegant information can be found in all sorts of applications, from weather apps to tax preparation software. Conversely, sparse and awkward information may be found in even the simplest of experiences. The difference between elegant and awkward is frequently less a matter of what than where.
Similarity
Planes Trains Automobiles
Planes TRAINS Automobiles
Planes Trains Automobiles Rocket Goats
As you can see, introducing any new element with a similar treatment maintains the group. Rocket goats may not be a method of transportation, but the example implies that they are. Make something look similar to something else, and users will assume that it is.
Pitfalls of Similarity
Now the button “Detonate Explosive” sets the context for “View Cat Photo.” Certainly, this button order makes one pause for at least a moment.
Important behaviors command separation within an interface. Checkout buttons, delete functions, and quit without saving are candidates for such treatment.
Gestalt grouping transforms random patterns into designed experiences, connecting the detached and highlighting the overlooked. It makes the new feel familiar. Handle gestalt grouping wisely, and you will be in good company.
Selective Perception
Hiawatha Service 332 bypasses the steady state of arterial road traffic between Milwaukee and Chicago. In 89 minutes, the train’s riders depart Miltown’s intermodal gateway and eventually find themselves in the heart of Chicago’s Union Station. Long an early morning refuge for blurry-eyed salespeople and late-night party-goers, Hiawatha attracts a wide assortment of professions, cultures, and hangovers. However, each rider’s trip is unique, because each is a selective perception.
Commuters and tourists alike fall asleep in peaceful unison within minutes of the train’s departure. Gaping mouths and contorted postures fill each carriage like dozens of goldfish placed on blue fabric seats, as the constant hum of the track passes beneath. Dah-dunk. Dah-dunk. Dah-dunk. The rhythm lulls even the most caffeinated to sleep. Some riders doze motionless. Others roll and fidget. They close their eyes. They wear headphones. They curl into the fetal position and form makeshift pillows out of jackets and sweaters. Consciousnesses rise and fall, governed by the lucidity of dreams and the placement of armrests.
Our brains search for stimuli, be it while riding a train or viewing a mobile app. We seek the pleasant and avoid the unpleasant. If our seat is comfortable, we relax and slumber. If light gets in our eyes, we close the shade. Aggravation yields to comfort. Pleasure beats provocation. Social psychologists call this selective exposure, or the “confirmation bias.” We find the messages that confirm our beliefs rather than challenge them. We readily notice ads for products we already own. We eagerly recognize virtues in the political candidates we already support. We unhesitatingly accept compliments about things we already enjoy.
One glowing attribute casts a halo around all others. This halo effect affects everything from interpersonal relationships to international branding. We believe attractive people are also kind. We think profitable companies are also managed well. We will even defend our favorite brands by ignoring their competitors’ advertising.17 You can experience this phenomenon firsthand: persuade an iPhone user to switch to Android; convince a Ford truck owner to buy a Chevrolet; coax a Snow user to download Snapchat; cajole a Diet Coke drinker to order a Diet Pepsi. Such halos surround us, enveloping our decisions in predictable delusion.
Moreover, we erect psychological barriers to threatening stimuli. Smoking causes nearly one in five deaths.18 Texting causes one in four car accidents.19 Unprotected sex causes one in two unplanned pregnancies.20 Yet, even after being exposed to the dangers of smoking, texting while driving, and unprotected sex, a great number of us still smoke, text, and spend anxious moments awaiting the results of a test strip.
Psychological barriers affect the user experience of digital products, as well. A recent Pew Research report indicates that only 9% of social media users feel very confident that their records are private and secure.21 However, the user base of such apps continues to grow, totaling 69% of the American public.22 Users weigh the tradeoffs between privacy and utility, though risks and rewards are often perceived selectively. Likewise, as of January 2017, 4% of Samsung Galaxy Note 7 phones23—the defective and recalled devices that may spontaneously combust and burn their owners—have yet to be returned. Few user experiences carry such dire consequences, but we should note that even possible immolation is not a compelling enough argument to offset some people’s selective perception.
We avoid the pitfalls of selective perception by acknowledging them. Users can even benefit from their inability to fully perceive an experience, thereby focusing on the necessary.
We can also avoid the pitfalls of selective perception by recognizing its triggers. If you wish to buy a car, your perception will become focused on cars. Recognize that the ads and offers you will find are the result of this trigger, even though the information was always there, waiting for you to perceive it. That car commercial you just watched may be more of a selective perception than a true bargain.
As designers, we can intercept these triggers, thereby directing users to what is necessary. For example, imagine someone shopping for a train ticket. He wants to find the best price. His focus is on the ticket’s cost, perceiving it above all other stimuli. He may not notice a train’s departure time. Placing vital information, such as the departure time, next to the price helps the user avoid a costly mistake. He catches the error before it happens, because we anticipated his selective perception.
Like daybreak filtering through a window, perception highlights some stimuli while obscuring others. It sifts through countless sights, sounds, smells, tastes, and touches, transforming what we could experience into what we will experience. Perception shapes our world. Patterns emerge, messages take form, and users awaken.
Key Takeaways
Top-down processing occurs when people form perceptions based on prior experiences, general concepts, and expectations.
Bottom-up processing occurs when people form perceptions through sensory data.
Understanding how perception works helps improve user experiences.
A schema is a mental shortcut, a way to interpret incomplete information.
People have a strong schema for human faces.
Mental models include schemata, behaviors, and outcomes.
People form mental models based on our related, past experiences.
JNDs (just noticeable differences) describe the smallest detectable difference between two levels of stimuli.
Marketers use JNDs to determine pricing and discounts.
Small multiples demonstrate differences among similar items.
People notice the relative change in stimuli, not the absolute.
Gestalt grouping laws describe how people perceive objects as organized patterns.
We perceive objects placed next to one another as a group.
When objects are similar to one another, we often perceive them as a group.
Selective perception leads people to seek the pleasant and avoid the unpleasant.
Halo effects influence our perception by assigning the value of a known attribute to an unknown attribute.
People erect psychological barriers to threatening stimuli.
We can also avoid the pitfalls of selective perception by acknowledging it and recognizing its triggers.
Questions to Ask Yourself
Have I considered users’ existing mental models?
What gestalt groupings have I intentionally and accidentally created within the experience?
Does the placement of items within my design connote a relationship where none exists?
Does the similarity of items within my design connote a relationship where none exists?
Will users recognize critical information within the experience?
Are there any areas of the experience that users may find unpleasant or fearsome?